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DETAILED ACTION 
Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

Claims 1-10 and 32-35 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Szeliski et al. US patent no. 6, 157,747 in view of Luken US patent no. 5,923,334. 

-Regarding claim 1, Szeliski et al. teaches a computerized method for creating a three 
dimensional model from one or more image panoramas (see column 1, lines 6-11, column 4, 
lines 8-9 and 66-67): receiving one or more image panoramas representing a visual scene and 
having one or more objects (see fig. 6, column 5, lines 51-52); a directional vector indicating an 
orientation of the visual scene with respect to a reference coordinate system (see fig. 15, column 
22, lines 4-9), creating a three dimensional model of the visual scene from the transformed image 
panoramas using the reference coordinate system and comprising geometry information 
describing the one or more objects contained in the scene (see column 5, lines 7-35, column 9, 
lines 4-26), transforming the image panoramas such that the directional vectors are substantially 
aligned relative to the reference coordinate system (see figs. 11 and 15, column 21, lines 25-59, 
column 22, lines 2-35), and aligning the transformed image panoramas to each other (see figs. 1 
and 2B, column 9, lines 29-36). 
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Szeliski does not directly teach a method for determining a directional vector for each 
image panorama. However in similar art Luken teaches said method (see figs. 7-10, 14 and 17, 
column 12, lines 35-44). 

Therefore it would have been obvious to one skilled in the art to combine Szeliski with 
Luken, because it would provide an efficient system for generating and viewing three- 
dimensional panoramic images based environment maps, and offer an improved level of 
interactive graphical feedback (see column 3, lines 5-8). 

-Regarding claim 2, Szeliski teaches a method wherein the directional vector is 
determined based, at least in part, on instructions identifying elements of the image panorama 
received from a user (see column 8, lines 30-32, column 27, lines 64-66). 

-Regarding claims 3 and 4, Szeliski teaches a method wherein the instructions from the 
user identify two or more substantially parallel features in the image (see column 20, line 64- 
column 21, line 6 and lines 25-34). 

-Regarding claim 5, Szeliski teaches a method wherein the instructions from the user 
identifying a horizon line of the image panorama (see fig 4, column 9, lines 54-62). 

-Regarding claim 6, Szeliski teaches a method wherein the instructions comprise the 
identification of two or more areas of the image (see figs. 6 and 18, column 20, lines 49-50, 
column 27, lines 32-44), each area containing one or more elements and further comprising 
automatically identifying the two elements contained in the two or more areas (see fig. 6, column 
20, lines 49-50, column 32, lines 24-27). 
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-Regarding claim 7, Szeliski teaches a method further comprising using edge detection to 
automatically identify two elements (see fig. 32, column 30, lines 13-17). 

-Regarding claim 8, Szeliski teaches a method wherein the image panoramas are aligned 
relative to the reference coordinate system such that the directional vector is at least substantially 
parallel to one axis of the reference coordinate system (see figs. 11 and 15, column 21, lines 25- 
59, column 22, lines 2-35 [it is obvious that as the image rotates into alignment the directional 
vector will be parallel to one axis]). 

-Regarding claim 9, Szeliski teaches a method wherein the image panoramas are aligned 
relative to the reference coordinate system such that the directional vector is at least substantially 
orthogonal to one axis of the reference coordinate system (see figs. 11 and 15, column 21, lines 
25-59, column 22, lines 2-35 [it is obvious that as the image rotates into alignment the directional 
vector will be orthogonal to one axis]). 

-Regarding claim 10, Szeliski teaches a method wherein the image panoramas are aligned 
according to instructions received from a user (see column 8, lines 30-32, column 25, lines 64- 
66, column 27, lines 64-66). 

-Regarding claim 32, Szeliski et al. teaches a system for creating a three dimensional 
model from one or more image panoramas (see fig. 2B, column 4, lines 66-67, column 9, lines 4- 
26) comprising: means for receiving one or more image panoramas representing a visual scene 
having one or more objects (see column 4, lines 8-13 [captured = received images] and 66-67); 
means for allowing a user to interact with the system (see column 8, lines 30-32, column 25, 
lines 64-66, column 27, lines 64-66); means for aligning the image panoramas relative to each 
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other (see figs. 1, 2B, 1 1 and 15, column 9, lines 29-36, column 21, lines 25-59, column 22, lines 
2-35); and means for creating a three dimensional model from the aligned panoramas (see figs. 1 
and 2B, column 9, lines 29-36). 

Szeliski does not directly teach a method for determining a directional vector for each 
image panorama. However in similar art Luken teaches said method (see figs. 7-10, 14 and 17, 
column 12, lines 35-44). 

Therefore it would have been obvious to one skilled in the art to combine Szeliski with 
Luken, because it would provide an efficient system for generating and viewing three- 
dimensional panoramic images based environment maps, and offer an improved level of 
interactive graphical feedback (see column 3, lines 5-8). 

-Regarding claim 33, Szeliski teaches a system wherein the input images comprise two- 
dimensional images (see figs. 1 and 2B, [\ (0 . k) are two dimensional]). 

-Regarding claim 34, Szeliski teaches a system wherein the input images comprise three- 
dimensional images including geometry information (see fig. 2B, column 5, lines 7-35, column 
9, lines 4-26). 

Regarding claim 35, Szeliski teaches a system wherein the image panoramas are aligned 
according to instructions received from a user (see figs. 1 and 2B, column 8, lines 30-32, column 
25, lines 64-66, column 27, lines 64-66). 

Claims 13-21, 30, 31 and 37 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Szeliski et al. US patent no. 6,157,747 in view of Blank US patent no. 5,469,536.13. 
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-Regarding claim 13, Szeliski does not directly teach a method comprising receiving an 
edit to color information associated with the objects of the image. However in similar art Blank 
teaches said method (see column 6, lines 29-47). Therefore it would have been obvious to one 
skilled in the art to combine Szeliski with Blank, because it would enable the user to quickly and 
efficiently modify or enhance the appearance of an image to desired goal (see column 6, lines 
23-28). 

-Regarding claim 14, Szeliski does not directly teach a method comprising receiving an 
edit to alpha information associated with the objects of the image. However in similar art Blank 
teaches said method (see fig. 11, column 20, lines 61-66). Therefore it would have been obvious 
to one skilled in the art to combine Szeliski with Blank, because it would enable the user to 
quickly and efficiently modify or enhance the appearance of an image to desired goal (see 
column 6, lines 23-28). 

-Regarding claim 15, Szeliski does not directly teach a method comprising receiving an 
edit to depth information associated with the objects of the image. However in similar art Blank 
teaches said method (see column 3, lines 59-67, column 13, lines 8-16). Therefore it would have 
been obvious to one skilled in the art to combine Szeliski with Blank, because it would enable 
the user to quickly and efficiently modify or enhance the appearance of an image to desired goal 
(see column 6, lines 23-28). 

-Regarding claim 16, Szeliski does not directly teach a method comprising receiving an 
edit to geometry information associated with the objects of the image. However in similar art 
Blank teaches said method (see column 47, lines 11-20). Therefore it would have been obvious 
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to one skilled in the art to combine Szeliski with Blank, because it would enable the user to 
quickly and efficiently modify or enhance the appearance of an image to desired goal (see 
column 6, lines 23-28). 

-Regarding claim 17, Szeliski does not directly teach a method comprising: providing a 
user with an interactive drawing tool that specifies edits for one or more objects of the image; 
and receiving the edits made by the user using the interactive drawing tool. However in similar 
art Blank teaches said method (see column 21, lines 18-23, column 28, lines 28-35). Therefore it 
would have been obvious to one skilled in the art to combine Szeliski with Blank, because it 
would enable the user to quickly and efficiently modify or enhance the appearance of an image 
to desired goal (see column 6, lines 23-28). 

-Regarding claims 18 and 37, Szeliski does not directly teach a method wherein the 
interactive drawing tool (editing tool) is one of an extrusion tool, a ground plane tool, a depth 
chisel tool or a non-uniform rational B-spline tool. However in similar art Blank teaches said 
method (see column 8, lines 46-52, column 34, lines 22-27). Therefore it would have been 
obvious to one skilled in the art to combine Szeliski with Blank, because it would enable the user 
to quickly and efficiently modify or enhance the appearance of an image to desired goal (see 
column 6, lines 23-28). 

-Regarding claim 19, Szeliski does not directly teach a method wherein the interactive 
drawing tool specifies a selected value for depth for objects of the image. However in similar art 
Blank teaches said method (see column 21, lines 14-27, column 22, lines 35-62). Therefore it 
would have been obvious to one skilled in the art to combine Szeliski with Blank, because it 
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would enable the user to quickly and efficiently modify or enhance the appearance of an image 
to desired goal (see column 6, lines 23-28). 

-Regarding claim 20, Szeliski does not directly teach a method wherein the interactive 
drawing tool incrementally adds to the depth for objects of the image. However in similar art 
Blank teaches said method (see column 34, lines 8-11). Therefore it would have been obvious to 
one skilled in the art to combine Szeliski with Blank, because it would enable the user to quickly 
and efficiently modify or enhance the appearance of an image to desired goal (see column 6, 
lines 23-28, column 48, lines 26-39). 

-Regarding claim 21, Szeliski does not directly teach a method wherein the interactive 
drawing tool incrementally subtracts from the depth for objects of the image. However in similar 
art Blank teaches said method (see column 34, lines 22-26). Therefore it would have been 
obvious to one skilled in the art to combine Szeliski with Blank, because it would enable the user 
to quickly and efficiently modify or enhance the appearance of an image to desired goal (see 
column 6, lines 23-28, column 48, lines 26-39). 

-Regarding claim 30, Szeliski does not directly teach a method wherein instructions are 
received using an interactive drawing tool. However in similar art Blank teaches said method 
(see column 18, lines 22-24, column 21, lines 18-23). Therefore it would have been obvious to 
one skilled in the art to combine Szeliski with Blank, because it would enable the user to quickly 
and efficiently modify or enhance the appearance of an image to desired goal (see column 6, 
lines 23-28, column 48, lines 26-39). 
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-Regarding claim 31, Szeliski does not directly teach a method wherein the interactive 
drawing tool is used to identify four or more features common to the two or more image 
panoramas. However in similar art Blank teaches said method (see fig. 3d, column 11, lines 4-27, 
column 18, lines 22-24, column 21, lines 18-23 [center, edge, color, hue, size, position, texture]). 
Therefore it would have been obvious to one skilled in the art to combine Szeliski with Blank, 
because it would enable the user to quickly and efficiently modify or enhance the appearance of 
an image to desired goal (see column 6, lines 23-28, column 48, lines 26-39). 

Claim Rejections - 35 USC § 102 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

Claims 11, 12, 22-29 and 36 are rejected under 35 U.S.C. 102(b) as being anticipated by 
Szeliski et al. US patent no. 6,157,747. 

-Regarding claim 11, Szeliski et al. teaches a computerized method of interactively 
editing objects in a panoramic image (see column 1, lines 6-11, column 4, lines 8-9 and 66-67, 
column 8, lines 30-32, column 27, lines 64-66) comprising: receiving an image panorama 
representing a visual scene (see fig. 3, column 5, lines 46-47, column 9, lines 55-62), the image 
panorama having one or more objects and a point source (see figs. 3, 4 and 6, column 5, lines 46- 
52); creating a three dimensional model of the visual scene using features of the visual scene and 
the point source (see fig. 2B, column 4, lines 66-67, column 9, lines 4-26); receiving an edit to 
one or more of the objects in the panorama (see fig. 1, column 7, lines 1 1-20); transforming the 
edit relative to a viewpoint defined by the point source (see column 10, lines 27-38, column 11, 
lineslO-21); and projecting the transformed edit onto the objects (see figs 2B-8, column 3, lines 
58-61, column 31, lines 44-51). 
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-Regarding claim 12, Szeliski teaches a three-dimensional model (see figs. 2B, 27, 30 and 
32, column 28, lines 29-31, column 29, lines 58-61), three-dimensional models inherently 
contain one or more of depth information and geometry information. 

-Regarding claim 22, Szeliski et al. teaches a method for projecting texture information 
onto a geometric feature within an image panorama (see fig. 2B, column 3, lines 58-61, column 
27, lines 34-55) comprising: receiving instructions from a user identifying a three-dimensional 
geometric surface within an image panorama (see column 7, lines 10-20), the image panorama 
containing features having one or more textures (see column 5, lines 27-35); determining a 
directional vector from the three-dimensional geometric surface (see figs. 2B and 15, column 22, 
lines 4-31); creating a geometric model of the image panorama based at least in part on the three- 
dimensional geometric surface and the directional vector (see figs. 2A-B, column 7, lines 25-33, 
column 9, lines 4-26, column 22, lines 4-31); and applying the one or more textures to the 
features in the image panorama based on the geometric model (see fig. 2B, column 9, lines 29- 
31, column 32, lines 22-29 and 41-43). 

-Regarding claim 23, Szeliski teaches a method wherein the instructions are received 
using an interactive drawing tool (see column 27, line 64-colum 28, line 2). 

-Regarding claim 24, Szeliski teaches a method wherein the three-dimensional geometric 
surface is one of a floor, a wall, or a ceiling (see figs. 33 and 38, column 30, lines 44-45). 

-Regarding claim 25, Szeliski teaches a method wherein the directional vector is 
orthogonal to the planar surface (see fig. 15, column 15, lines 64-66, column 22, lines 4-6). 
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Regarding claim 26, Szeliski teaches a method wherein the geometric model comprises 
depth information (see figs. 2B, 27 and 30, column 28, lines 29-33). 

-Regarding claim 27, Szeliski teaches a method wherein the texture information 
comprises color information (see column 28, lines 13-18). 

-Regarding claim 28, Szeliski teaches a method comprising environment based texture 
mapping (see fig. 2B, column 5, lines 27-30, column 9, lines 12-26), in said method the texture 
information inherently includes luminance information (see figs. 33-38). 

-Regarding claim 29, Szeliski et al. teaches a computerized method for creating a three- 
dimensional model of a visual scene from a set of image panoramas (see fig. 2B, column 4, lines 
66-67, column 9, lines 4-26), receiving multiple image panoramas (see column 4, lines 8-13 
[captured = received images]); arrange each image panorama to a common reference system (see 
figs. 37-38, column 31, lines 53-67); receiving information identifying features common to two 
or more of the arranged panoramas (see column 20, lines 49-50 and 64-66); aligning the two or 
more image panoramas to each other using the identified features (see fig. 18, column 24, lines 
33-44, column 32, lines 24-29); and creating a three-dimensional model from the aligned image 
panoramas (see figs. 2B, column 4, lines 8-30 and 66-67, column 9, lines 4-26). 

-Regarding claim 36, Szeliski et al. teaches a system for interactively editing objects in a 
panoramic image (see column 1, lines 6-11, column 4, lines 8-9 and 66-67, column 8, lines 30- 
32, column 27, lines 64-66) comprising: a receiver for receiving one or more image panoramas 
representing a visual scene (see fig. 3, column 5, lines 46-47, column 9, lines 55-62) having one 
or more objects and a point source (see figs. 3, 4 and 6, column 5, lines 46-52); a modeling 
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module for creating a three dimensional model of the visual scene (see figs. 2B, 27, 30 and 32, 
column 4, lines 66-67, column 9, lines 4-26, column 28, lines 29-31, column 29, lines 58-61) 
including depth information describing the objects (three-dimensional models inherently contain 
depth information), one or more interactive editing tools for providing an edit to one or more 
objects in the panorama (see fig. 1, column 7, lines 11-20, column 8, lines 30-32); a 
transformation module for transforming the edit relative to a viewpoint defined by the point 
source (see figs. 2B and 8, column 10, lines 27-38, column 11, lineslO-21); and a rendering 
module for projecting the transformed edit onto the objects (see figs. 1 and 2A-B, column 3, 
lines 58-61, column 31, lines 44-51). 

Double Patenting 

A rejection based on double patenting of the "same invention" type finds its support in 
the language of 35 U.S.C. 101 which states that "whoever invents or discovers any new and 
useful process ... may obtain a patent therefor..." (Emphasis added). Thus, the term "same 
invention," in this context, means an invention drawn to identical subject matter. See Miller v. 
Eagle Mfg. Co., 151 U.S. 186 (1894); In re Ockert, 245 F.2d 467, 114 USPQ 330 (CCPA 1957); 
and/« re Vogel, All F.2d 438, 164 USPQ 619 (CCPA 1970). 

A statutory type (35 U.S.C. 101) double patenting rejection can be overcome by 
canceling or amending the conflicting claims so they are no longer coextensive in scope. The 
filing of a terminal disclaimer cannot overcome a double patenting rejection based upon 35 
U.S.C. 101. 

Applicant is advised that should claim 3 be found allowable, claim 4 will be objected to 
under 37 CFR 1.75 as being a substantial duplicate thereof. When two claims in an application 
are duplicates or else are so close in content that they both cover the same thing, despite a slight 
difference in wording, it is proper after allowing one claim to object to the other as being a 
substantial duplicate of the allowed claim. See MPEP § 706.03(k). 
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Conclusion 

The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

US patent no. 7,046,840 B2 
US patent no. 6,636,216 Bl 
US patent no. 6,246,412 Bl 
US Pub. no. 2003/0095131 Al 
US Pub. no. 2003/0063089 Al 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Enrique L Santiago whose telephone number is (571) 272-7648. 
The examiner can normally be reached on Monday to Thursday from 6:30 A.M. to 4:30 P.M. 

If attempts to reach the examiner by telephone are unsuccessful, the examinees 
supervisor, Mark K. Zimmerman whose telephone number is (571) 272-7653, can be reached on 
Monday to Friday from 7:00 A.M. to 3:30 P.M. 

Any response to this action should be mailed to: 

Commissioner of Patents and Trademarks 
Washington, D.C. 20231 

Faxed to: 571-273-8300 

Hand carried deliveries must be made to the Customer Service Window (now located at 
the Randolph Building, 401 Dulany Street, Alexandria, VA 22314) 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
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may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
Enrique L. Santiago 

September 6, 2006 # < 




